Sample size selection in optimization methods for machine learning
نویسندگان
چکیده
منابع مشابه
Sample size selection in optimization methods for machine learning
This paper presents a methodology for using varying sample sizes in batch-type optimization methods for large scale machine learning problems. The first part of the paper deals with the delicate issue of dynamic sample selection in the evaluation of the function and gradient. We propose a criterion for increasing the sample size based on variance estimates obtained during the computation of a b...
متن کاملStochastic Methods For Optimization and Machine Learning
In this project a stochastic method for general purpose optimization and machine learning is described. The method is derived from basic information-theoretic principles and generalizes the popular Cross Entropy method. The effectiveness of the method as a tool for statistical modeling and Monte Carlo simulation is demonstrated with an application to the problems of density estimation and data ...
متن کاملOptimization Methods for Large-Scale Machine Learning
This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machi...
متن کاملLine search methods with variable sample size for unconstrained optimization
Minimization of unconstrained objective function in the form of mathematical expectation is considered. Sample Average Approximation SAA method transforms the expectation objective function into a real-valued deterministic function using large sample and thus deals with deterministic function minimization. The main drawback of this approach is its cost. A large sample of the random variable tha...
متن کاملModern Machine Learning for Automatic Optimization Algorithm Selection
Optimization software is commonly used to solve simulation-based problems such as optimal design and control, model parameter estimation, and best/worst-case scenario identification. While the value of such software is widely recognized, user feedback indicates that these tools are difficult for nonexperts to use. In particular, users are unfamiliar with the details of the plethora of available...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Mathematical Programming
سال: 2012
ISSN: 0025-5610,1436-4646
DOI: 10.1007/s10107-012-0572-5